A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms

The data augmentation (DA) algorithm is a widely used Markov chain Monte Carlo (MCMC) algorithm that is based on a Markov transition density of the form p(x|x′) = ∫ Y fX|Y (x|y)fY |X(y|x) dy, where fX|Y and fY |X are conditional densities. The PX-DA and marginal augmentation algorithms (Liu and Wu, 1999; Meng and van Dyk, 1999) are alternatives to DA that often converge much faster and are only slightly more computationally demanding. The transition densities of these alternative algorithms can be written in the form pR(x|x) = ∫ Y ∫ Y fX|Y (x|y′)R(y, dy)fY |X(y|x) dy, where R is a Markov transition function on Y. We prove that whenR satisfies certain conditions, the MCMC algorithm driven by pR is at least as good as that driven by p in terms of performance in the central limit theorem and in the operator norm sense. These results are brought to bear on a theoretical comparison of the DA, PX-DA and marginal augmentation algorithms. Our focus is on situations where the group structure exploited by Liu and Wu (1999) is available. We show that the PX-DA algorithm based on Haar measure is at least as good as any PX-DA algorithm constructed using a proper prior on the group. AMS 2000 subject classifications. Primary 60J27; secondary 62F15 Abbreviated title. A Comparison of Data Augmentation Algorithms